Reconstructing Complex Surfaces from Multiple Stereo Views

We present a framework for 3-D surface reconstruction that can be used to model fully 3-D scenes from an arbitrary number of stereo views taken from vastly different viewpoints. This is a key step toward producing 3-D world-descriptions of complex scenes using stereo and is a very challenging problem: real-world scenes tend to contain many 3-D objects, they do not usually conform to the 2-1/2-D assumption made by traditional algorithms, and one cannot take it for granted that the computed 3-D points can easily be clustered into separate groups. Furthermore, stereo data is often incomplete and sometimes erroneous, which makes the problem even more difficult. By combining a particle-based representation, robust fitting, and optimization of an image-based objective function, we have been able to reconstruct surfaces without any a priori knowledge of their topology and in spite of the noisiness of the stereo data. Our current implementation goes through three steps—initializing a set of particles from the input 3-D data, optimizing their location, and finally grouping them into global surfaces. Using several complex scenes containing multiple objects, we demonstrate its competence and ability to merge information and thus to go beyond what can be done with conventional stereo alone.